REGULARITY OF SETS WITH CONSTANT HORIZONTAL NORMAL 

IN THE ENGEL GROUP 
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Abstract. In the Engel group with its Carnot group structure we study subsets of locally 
finite subRiemannian perimeter and possessing constant subRiemannian normal. 

We prove the rectifiability of such sets: more precisely we show that, in some specific 
coordinates, they are upper-graphs of entire Lipschitz functions (with respect to the Eu- 
clidean distance). However we find that, when they are written as intrinsic upper-graphs 
with respect to the direction of the normal, then the function defining the set might even 
fail to be continuous. Nevertheless, we can prove that one can always find other horizontal 
directions for which the set is the upper-graph of a function that is Lipschitz-continuous 
with respect to the intrinsic distance (and in particular Holder-continuous for the Euclidean 
distance). 

We further discuss a PDE characterization of the class of all sets with constant hori- 
zontal normal. 

Finally, we show that our rectifiability argument extends to the case of filiform groups 
of the first kind. 
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1. Introduction 

Recent years have witnessed an increasing interest in Geometric Analysis of Metric Spaces. 
A particular role has been played by the class of Carnot groups endowed with subRieman- 
nian distances. In this setting, both translations and dilations are present, hence the theory 
of differentiation generalizes. Many notions from Analysis and Geometry have been investi- 
gated in subRiemannian Carnot groups. Function Theory has been a fruitful study. There 
have been several fundamental results in the study of maps such as Lipschitz, Sobolev, 
quasiconformal, and bounded variation. Another subject of large interest has been Geo- 
metric Measure Theory. Minimal surfaces, sets with finite perimeter, currents, and rec- 
tifiable sets have received particular attention. As a source of reference, we point out to 
[(^DPTOTl IVitnSllLDTn] . and the references therein. 

In this paper we intend to contribute to the study of the regularity of some particular 
class of hypersurfaces in a specific Carnot group, namely the Engel Lie group, which is 
the lower-dimensional Carnot group of step 3. It is the simplest example for which there 
is lack of rectifiability results. We consider sets that are the generalization of half-spaces, 
i.e., they have constant horizontal normal. Such sets are doubly important: they appear as 
tangents of sets with locally-finite horizontal perimeter, and their boundaries are examples 
of minimal hyper-surfaces that can be written as entire graphs with respect to the group 
structure. For such reasons, it would be fundamental to understand whether such sets are, 
in some sense, rectifiable. 

We shall give an ambivalent study of the regularity of such constant normal sets. Briefly, 
we will show that such sets are rectifiable (even in the Euclidean sense) but not necessarily 
intrinsic Lipschitz graphs in the direction of the normal. In fact, there is an example of 
a set with constant horizontal normal such that, if one writes the set as a upper-graph in 
the direction of the normal, the function giving the graph is not continuous. These sets are 
however intrinsic Lipschitz graphs in other horizontal directions. 

Such (horizontal) intrinsic graphs have natural parametrizations and have been extend- 
edly considered in the theory of rectifiable subsets of Carnot groups, as examples in |KSC041 
IFSSC06[ IASCV06[ IBASCV071 IMSCV081 iBVlOl iBCTOl IBSClOl IFSSCllj . It is not clear 
whether a rectifiability result can be expected to hold for sets with constant horizontal 
normal in general Carnot groups. Nor we have examples of finite-perimeter sets for which 
it fails that, at almost every point of the boundary, the tangent is a half-space. 

1.1. Main setting, terminology, and previous contributions. Let G be a Carnot 
group (see |AKLD09] for definitions). Let g be the Lie algebra of the left-invariant vector 
fields in G. By definition, q is stratified. We denote by Vi the first stratum (also known as 
horizontal layer). 

A subset E oi a Carnot group is said to have locally finite horizontal perimeter if, for any 
X G Vi, the distribution XIe is a Radon measure. Caccioppoli and De Giorgi introduced 
these sets (in the Euclidean space) for the study of minimal hyper-surfaces. The reason for 
doing so is the good behavior of the perimeter, which is the total mass of the vector-valued 
measure whose components are obtained by differentiating l^; in the directions of a fixed 
basis of Vi. In fact, the perimeter is lower semicontinuous and induces a locally compact 
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topology on the class of finite-perimeter sets. Hence, it becomes easy to show existence of 
minimal surfaces. 

A set E in a Carnot group G is said to have constant horizontal normal if there exists a 
horizontal left-invariant vector field X on G and there exists a decomposition RX © of 
Vi with the following property: 

• the distributional derivative XIe of the characteristic function l^; of in the 
direction of X is a positive Radon measure; 

• for all Y , the distribution YIe vanishes. 

One should notice that the space is uniquely defined by E, unlike the vector X. 
However, if we fix a scalar product on Vi and require that X is a unit vector orthogonal to 
V^, then X is unique and it is called the normal of E. 

In |FSSC03] , the three authors extended a result of De Giorgi by proving rectifiability of 
sets with locally finite horizontal perimeter in Carnot groups of step 2. Following De Giorgi's 
strategy, they obtained this result, by showing that almost every tangent is a set of constant 
normal and that constant-normal sets are in fact half-spaces. Alas, they noticed that this 
latter fact does not hold in higher-step Carnot groups. In fact, in |FSSC03t Example 3.2], 
they gave the first explicit example of a subset of the Engel group with constant normal 
that is not a half-space. More examples have then been given in |AKLD09] and we will be 
adding some more in the present work. 

In jBASCVOTj . the three authors showed that sets with constant horizontal normal are 
calibrated sets: the calibration that they used is the scalar product with the normal. This 
calibration method implies that, in any Carnot group, boundaries of sets with constant 
horizontal normal are minimal surfaces, just as it happens in the Euclidean framework. 
Guided by the Euclidean experience, one could expect fairly good results on the smoothness 
of calibrated sets: the classical regularity theory for minimal sets in (R" indeed argues, in 
its key steps, as follows. First write the set, locally around a point where we have a tangent 
plane, as a graph on the tangent plane; then prove that the normal is Holder continuous. 
It is now crucial the fact that, in this Euclidean setting, one can further improve regularity 
to (for the original proofs see |DG61j and |Giu84] ). 

Going back to the subRiemannian framework, one can write any constant-normal set as 
an intrinsic upper-graph in the direction of the normal. Such graphs have been considered in 
|KSC04[ IFSSC061 IASCV06[ IBASCV07[ IMSCVOSl lBVTOl iBCTOl IBSClOi IFSSCllj and give 
canonical parametrizations. By the work of [MVllj . it was expected that, if the uppergraphs 
of a function were giving sets with controlled normal (e.g., constant), then the function had 
some regularity, as it happens in the classical Euclidean case that we mentioned before. 

We shall give examples in the Engel group of sets with constant horizontal normal where 
the function is not even continuous. Here the choice of a specific normal direction (namely, 
of a scalar product on Vi) will be crucial. On the other hand we can prove that, by choosing 
other horizontal directions, we can express the set as upper-graph of a function taking values 
in the new direction, and this function is intrinsically Lipschitz continuous, in particular 
Holder continuous. 
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1.2. Overview of results. We recall now the definition of the Carnot group of interest to 
us, the Engel group. The Engel algebra is the Lie algebra generated, as vector space, by 
four vectors Xi, X2, X^, X4, with relations 

(1.1) [Xi,X2] = Xs and [Xi,Xs]=Xi, 

[Xi,X4] = [X2,X4] = [X2,X3] = [X3,X4] = 0. 

Such an algebra is nilpotent of step 3 and stratified by the strata 

Vi := RXi © RX2, V2 := RX3, V3 := RX4. 

The Engel group is defined as the unique connected and simply connected Lie group with 
the Engel algebra as Lie algebra. Through the paper we denote by G such a group. 

We endow the Engel group G with some Haar measure voIq- Given a measurable set 
E C G and a left-invariant vector field X G Lie{G), we write 

XIe > ( resp. XIe = 0) 

if, for aU (/) G C^{G) with > 0, 

- X(j) dvole > ( resp. / Xcp dvole =0). 

Je Je 

Since the flow of a left-invariant vector field is a right translation, then the flow of such a 
vector field preserve the Haar measure, which on a nilpotent group is always biinvariant. In 
other words, any element of the Lie algebra is a divergence-free vector field on the manifold 
G, endowed with a Haar measure voIq- 

Definition 1.2 (Constant horizontal normal). Let Vi be the first stratum of Lie{G). Fix 
a scalar product (-j-) on Vi. A set i? C G is said to have constant horizontal normal 
X £ Lie{G) \i X eVi, X1e>Q, and 

Y G Vi, {X\Y) = ^ YIe = 0. 

Regarding the next definition, recall that, being connected, simply connected, and nilpo- 
tent, the Engel group G is diffeomorphic to Lie{G), via the exponential map, and so is 
diffeomorphic to R^ . 

Definition 1.3 (Lipschitz domain). A set C G is called a Lipschitz Euclidean domain if, 
for one (and thus for all) diffeomorphisms / : G — )■ R^, the set f{E) is a Lipschitz domain 
of R^. Namely, f{E) is an open set and any point on the boundary has a neighborhood in 
which the set can be described as the upper-graph of a Lipschitz map of three variables. 

Our first result is the following. 

Theorem 1.4. If E is a subset of the Engel group G that has constant horizontal normal, 
then there exists a Lipschitz domain E C G that is equivalent to E, i.e., it is such that 
vol{EAE) = 0. 

The strategy of the proof is as follows. Let Yi = X he the normal of E. Take Y2 G Vi 
with {X\Y2) = 0. Hence lil^ > and Y2IE = 0. Using a result of jAKLD09l Proposition 
4.7], we get 1^3,14 G Lie{G) such that Yi,Y2,Ys,Y4 form a basis, Y^Ie > 0, and Y^Ie > 0. 
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Take the Lebesgue representative E of E. The set E wih have the property that, for ah 

4 

i= 

G R : pexpitZ) e E} = (0,+cx)). 



p € dE and for ah Z = '^j^j with > 0, one has that 



In other words, the 'cone' 



Cp 



pexp I ^ajYj j : aj>0 



does not intersect E. Finally, a standard cone criterion gives the Lipschitz regularity of dE. 

In the proof we get a cone Cp that in exponential coordinates of second kind is constant, 
i.e., Cp is a left translation of Cq, for p,q G dE. Thus we might conclude that dE is an 
entire Lipschitz graph. 

Theorem 1.5. Let G be the Engel group. Let ^' : — > G 6e the exponential coordinates 
of the second kind. There exists L > such that, for all horizontal vector X , there exists a 
basis wi, . . . ,W4, G R'^ with the following property. If E C G is any subset that has constant 
horizontal normal X, then there exists an L- Lipschitz map /i : R'^ — > R such that E is 
equivalent to 

{^{aiwi + . . . + a^w^) : ai, a2, as € R, 04 > /i(ai, 02, 03)}- 

Our rectifiability argument can be extended to all those filiform groups that are generated 
by two vectors X and Y and for which Y and all the iterated commutators of X and Y are 
in the center. These are the so-called "filiform groups of the first kind". See Section [5] for 
a discussion on filiform group and the proof of the result. 

Going back to the Engel group, we now turn our attention to the expression of E as an 
"algebraically intrinsic horizontal graph" (see Section This means that we express E 
as union of half-fiow-lines in a horizontal direction. More precisely, let X be a horizontal 
direction. Let W he a subgroup that is complementary to exp(RX). Let T : W ^ R he 
any function. We say that E is an intrinsic horizontal upper-graph if 

E = {wexp{tX) ■.weW,t> T{w)}. 

One should observe that the point w exp{tX) is the fiow from w for time t in the direction 
X. Understanding the regularity of T is then our next task. 

The most natural horizontal direction to use is the direction of the normal. Nevertheless 
we find that 

(i) : (see Theorem l4.ip There are examples of subsets E C G with constant horizontal 
normal X2 and with the property that, when they are written as upper- graphs in 
the direction of X2, the function for which they are upper graphs is not continuous. 

(ii) : (see Theorem 14. 5p Let £^ C G be an arbitrary set having constant horizontal 
normal X. Whenever E is written as intrinsic horizontal upper-graph using a hori- 
zontal direction Y with {X, Y) ^ and X not parallel to Y, then the function for 
which it is upper-graph is Lipschitz continuous with respect to the intrinsic Carnot- 
Caratheodory distance on G. In particular the function is Holder continuous for the 
Euclidean distance. 
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We note that the 'Euchdean Lipschitz continuity' obtained in Theorem 11.51 requires the 
expression of -E as a graph with respect to a non-horizontal direction, see Theorem l4.31 This 
confirms that the natural notion (intruduced by Franchi, Serapioni, and Serra Cassano in 
|FSSC06j . see also [FSSCllj ) of Lipschitz continuity to be used for intrinsic horizontal 
graphs in the subRiemannian context is the one with respect to the intrinsic distance. 

To complete the description of sets with constant horizontal normal in the Engel group, 
in exponential coordinates of second kind we are able to give a characterization of any such 
set as upper-graph of a function that satisfies a partial differential inequality. Roughly 
speaking we can prove that a set has finite perimeter and constant horizontal normal X2 if 
and only if it is of the form {x2 > ^(3:3, 3:4)} for a BV function G : — )> IR that satisfies 
the following partial differential inequality: for all h G C^(G) such that /i > 0, it holds 

(1.6) {{d:iG,h)f + 2{diG,h){C^,h) <0. 

Here denotes the Lebesgue measure on (R^ and ( , ) denotes the pairing of distributions 
and smooth test functions. The precise statement of the result (see Proposition I3.2ip 
requires to extend the target of G to {—00} U IR U {-|-oo}. Minimal graphs of functions 
that also assume the values -|-cxd and —00 have already appeared in the Euclidean setting. 
Namely, Mario Miranda considered them in the solution of the Dirichlet's problem for the 
minimal surfaces equation, see |Mir77j and |Giu84|, Chapter 16]. 

1.3. Acknowledgements. Both authors would like to thank ETH Zurich for its supporting 
research environment while part of this work was conducted. This paper has benefited from 
numerous discussions with Luigi Ambrosio, Bruce Kleiner, Raul Serapioni, Francesco Serra 
Cassano, and Davide Vittone. Special thanks go to them. 

2. Getting more monotone directions 

Let S be a subset of the Engel group G that has constant horizontal normal X S Lie{G). 
Let y be a vector in Vi that is orthogonal to X. Notice that the line UY is independent 
from the scalar product chosen on Vi. Now we face a dichotomy: either Y is parallel to the 
vector X2 of the definition of the Lie algebra representation (jl.ip of Lie{G), or not. In the 
second case, we show that we can change the basis of Lie{G) and assume that Y = Xi. 

The case Y = X2 is easy to handle and in fact we show that E is (equivalent to) a 
half-space. The case Y = Xi is more complex and in this case it is not true that S is a 
half-space, as it was previously shown in |FSSC03] . 

2.1. Easy case: sets with normal Xi. Let G be the Engel group whose Lie algebra is 
generated by Ai, A2, A3, A4 with relations (|l.ip . 

Lemma 2.1. Let E C G. Assume that XiIe > and X2IE = 0. Then E is a vertical 
half-space. 

For the definition and other characterization of half-spaces see |AKLD09] . We make use 
of the following property of stability of monotone directions. 

Proposition 2.2 ( |AKLD09t Proposition 4.7]). Let X,Y e Lie{G) and E C G. Assume 
that XIe = and YIe > 0. Then (Adexp(x)^)li? > 0. 
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Recall that 

Ade,p(x)^ = e^'^^y = Y + [X,Y] + [X,Y]\, 

in a 3-step group. 

Proof of Lemma \2.1[ Applying Proposition 12.21 with X = tX2, Y = Xi, and t € K, we get 
that the vector field 

Z := Ad,,p(jx,)^i = Xi + [tX2, Xi] + ^[tX2, [tX2,Xi]] = Xi - tX^ 

is such that ZIe > 0, for all t S R. Letting t — )■ +oo and t —oo, respectively, we get 
that both —X-^Ie > and X-^Ie > 0. Hence X^Ie = 0. Apply again the proposition with 
X = tXs, Y = Xi, and t e R. Thus the vector 

Z' := Adcxp(jX3)-'^i = -^1 + [tX^, Xi] = Xi - tXi 

is such that Z'Ie > 0, for all t G K. Arguing as before, we conclude that X^Ie = 0. 
By the BV characterization of vertical half- spaces, see |AKLD09t Proposition 4.4], we are 
done. □ 

2.2. Hard case: sets with normal X2- We first argue that if the normal is not Xi then 
we can assume that the normal is X2. 

Lemma 2.3. Let q be the Engel algebra with basis Xi, X2, X^, X4 and relations (jl.ip . Let 
X = aXi + (3X2 with a, /3 G R and a 7^ 0. Then there exists a Lie algebra endomorphism 
ip of Q such that TpXi = X. 

Proof. Define V' • ~^ by the property 

ipXi = aXi + 13X2 and ipX2 = X2, 

and 

= 0X3 and 4^X4 = 0^X4. 
It is straightforward Q to check that such a ^/^ is an isomorphism. □ 
By the above lemma, the following fact is immediate. 

Corollary 2.4. Let E be a subset of the Engel group G. Let X,Y G Vi C Lie{G) linearly 
independent. Assume XIe > and YIe = 0. Then there exists a basis Xi, X2, X^, X4 of 
Lie{G) with relations such that 

• either XiIe > and X2IE = 0, 

• or XiIe = and X21e > 0. 

In other words, we only need to study the cases where either E has normal Xi or it has 
normal X2. Since we already solved the first case, let us focus now on the second. 

If one applies Proposition l2.2l to the case of constant equal to X2, one obtains the following. 

^Here is the calculation: Tp[Xi,X2] = ipXs = 0X3 = a[Xi,X2] = [tpXi,i!X2] and ip[Xi,X3] = 1^X4 = 
0^X4 = a^[X^,X3] = [ipX^,i>X3]. 



8 



COSTANTE BELLETTINI AND ENRICO LE DONNE 



Lemma 2.5. Let E C G. Assume that X2IE > and XiIe = 0. Then, for all t G R, the 

vector 

Zt := X2 + tX-i + 
is such that ZiIe > 0. In particular, 

XaIe > and {X2 + 2X3 + 2X4)1e > 0. 

One should notice at this point that the four vectors 

Yi:=Xi, Y2:=X2, Y3 := X4, ^ := ^2 + 2X3 + 2X4 

forms a basis of Lie{G). Moreover, for all j = 1, ... ,4, we just proved that YjIe > 0. The 
next proposition will then conclude the proof of Theorem 11.41 

Proposition 2.6. Let G he any Carnot group. Let E C G. Let Yi, . . . ,Yn be a basis of 
Lie{G). Assume that YjIe > 0, for all j = 1, . . . ,n. Then E is equivalent to an (Euclidean) 
Lipschitz domain. 

We need to postpone the proof of Proposition 12.61 since we need some preliminaries for 
it. Namely, we need to choose a good representative for the set E. 

Proof of Theorem \1.4\ According to Corollarv l2.4| we might consider two cases: either the 
set E has normal Xi or it has normal X2. In the first case, Lemma |2 . 1 1 implies that E is a 
vertical half-space, and so a Lipschitz domain. 

If the normal is X2, Lemma 12.51 give us four linearly independent monotone directions, 
i.e., YjIe > 0. Hence, Proposition 12.61 concludes. □ 

3. Regularity of sets with normal X2 

In Section 13.11 we will show Theorem 11.41 by proving that the Lebesgue representative E 
of a set E having constant horizontal normal X2 satisfies a "cone criterion" and is therefore 
an Euclidean Lipschitz domain. 

We will then, in subsection 13.21 fix a model of the Engel group and show that in this 
model, by suitably rotating coordinates, E can actually be expressed as the subgraph of an 
entire Lipshitz function. In the last subsection 13.31 we characterize all sets having constant 
horizontal normal X2. 

3.1. Sets with normal X2 are Lipschitz domains. We need to choose a good repre- 
sentative for the set E. In fact we want to have an equivalent set E for which all line flows 
of Yj, j = 1, . . . ,n, meet E in a half-line. Such a fact will also be useful for writing dE as 
a graph. The fact that such graphs will be uniformly Lipschitz is because, by assumption, 
they avoid cones that are left translations of the same fixed cone. 

The strategy requires the study of the Lebesgue representative of our original set, which 
allows us to obtain monotonicity along every flow line on every direction Yj. Subsequently, 
we show that the topological boundary of this new set is locally a Lipschitz graph. 
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Recall that if X is a left-invariant vector field in a Lie group G, i.e., X € Lie{G), then its 
flow is a right translation. Namely, 

^x{p,t) =pexp{tX), Mp € G. 

Lemma 3.1. Let G he any Carnot group. Let E C G. Let X € Lie(G). Assume that 
XIe > 0. Then, for any t > 0, we have that (almost everywhere) it holds 1_e < ^E°^x{', i)- 
In particular, it is true that a.e. 

lEexp(X) < Ie- 

Proof. Being X a divergence free vector field on the manifold G, endowed with a Haar 
measure voIq , we can prove that: if n € -Lj'^^(G) satisfies Xu > in the sense of distributions, 
then, for all t > 0, it o ^ u volg-a.e. in G. The statement of the lemma then follows 

immediately. 

What we need to show is that, for any non-negative g € Cl{G), the map t ^ J^gu o 
voIg is increasing in t. Indeed, the semigroup property of the flow, and the fact 
that X is divergence-free yield 

/ fifti o + s) dvolc - / 5u o (ivole 

Jg J c 

= I ugo^x{-,-t- s)dYolQ - ugo^x{-,-t)dvolQ 
JG Jg 

= / ugo^x{^x{-,-s),-t)dvolc - u g o ^x{-, -t) dvolG 
Jg Jg 

= -s uX{go<^>x{-,-t))dvolc+o{s) 
Jg 

= s {go^x{-,-t))XudvolG+o{s), 
Jg 

which, recalling that Xu > 0, yields that t guo ^xi',t) dvolc is (weakly) increasing 
in t. □ 

Lemma 3.2. Let G be any Carnot group. Let E C G. Let Y'l,...,!^ S Lie{G). Assume 
that YjIe > 0, for all j = 1, . . . ,k. Then there exists E such that voIq (EAE) = and, for 
all p ^G and j = 1, . . . ,k, there exists T € [— oo, -|-oo] such that 

{t£R: pexp{tYj) e E} = {T,+oo). 

Proof. In a Carnot group, such as G, Haar measures are both left- and right-invariant. In 
this proof we will make use of the fact that voIq is right-invariant. Flows of left-invariant 
vector fields are right translations, thus isometries for such any right-invariant distance. The 
balls Br considered in this proof are to be understood with respect to a fixed right-invariant 
metric. 

Let E be the Lebesgue representative of E, i.e. the set of points having density 1: 

■r^o voIg (-Br (a;)) 
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By the Lebesgue-Besicovitch Differentiation Tlieorem, E and E agree volQ-a.e. We claim 
that E fulfils the requirements of Lemma[321 Indeed, what we need to prove is: let p £ E, X 
a left-invariant vector field such that XIe > 0: then for any t > the point y = pexp(tX) 
belongs to E. 

The vector field X is smooth and the fiow ^xi',t) is an isometry for the right-invariant 
metric, so it sends balls to balls of the same size. In the following denote by y the point 

^x(-, t)(p)- We assume p £ E, so lim — ^[ ^ = 1. By the invariance of voIq along 

r^O V0lQ{Br{p)) 

the fiow we have volQ{Br{p)riE) = voIq {Br{y) Ci E exp{X)) and volG(i?r(p)) = voIq (-Br(y))- 
With the aid of Lemma [XTl we then have voIq {Br (p) (1 E) < voIq {Br{y) fl E). Altogether 
we can write 

vo\G{Br{y)nE) ^ vol<,{Brip)nE) 

Yols {Br {y) ) - yoh{Br{p) ) 

Therefore we have 

^^.oHB{p)nE)^^ ^ ^^.oUB{y)nE)^ 

r^O V0lG{Br{p)) VoleC^rl?/)) 

and the lemma is proved. □ 

Observation 3.3. If Yi,...,Yk form a basis of Lie(G), then the set E is actually open. 
Indeed, let g be a point on the topological boundary of E and let us show that the upper 
density of £^ at a; is striclty less than one. Any direction in the convex envelope of some 
given monotone directions is in turn monotone, thus the whole cone Y obtained as convex 
envelope of li, ...,11- is made of monotone directions. Under the assumption that Yi, ...,Yk 
form a basis of Lie{G), this cone has non-empty interior. The complement of E is also a set 
with constant horizontal normal and contains a sequence of points qn converging to q. By 
means of lemma [3121 the cone qn — Y is all contained in the complement of E. The interior 
of Y is non-empty, so as qn ^ q the upper density oi E at q must be stricly less than 1. 

Hence, by Lemma 12.51 every set in the Engel group that has normal X2, has a represen- 
tative that is open and is satisfies the conclusion of Lemma 13.21 

Proof of Proposition \2.6[ By the lemma just proved, we can assume the following: let E C 
G, where G is a Lie group, let li, . . . , y„ be a basis of Lie{G) such that for all p G dE and 
J = 1, . . . , fc, we have that 

{t G R : pexp{tYj) e E} = (0, +00). 
We want to show that dE is locally a Lipschitz graph. 

Fix pq € dE. For all p € G consider the set 

Cp := jpexp |^X^aj>"jj : > 

Fix a (smooth) coordinate chart (p :U — ?> from a compact neighborhood of po- Since (p is 
smooth and Cp change smoothly in p, then, for all p £ U, the set (p{Cp) changes smoothly. 
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Thus one can find a fixed set C C (R" of the form 

C := \^ajVj : aj > o| , 

for some basis f i, . . . , of K", such that 

ip{dE)r\{x + C) = <D, yx£^{dE). 

Note that consequently we also have that Lp{dE) n (x — C) = 0. By a standard argument, 
e.g., see |AFPOO[ Theorem 2.61, page 82], one can write ip{dE) as a graph in any direction 
V (z C with respect to any hyperplane 11 such that 11 n (C U — C) =0. □ 

Observation 3.4. As a byproduct we get of course that the set E has rectifiable boundary. 

3.2. Further regularity in a model of the Engel group. We will fix, for this subsection 
and the next, the following model of the Engel group and work in it, proving that a set with 
constant horizontal normal is, in suitable coordinates, the subgraph of an entire Lipschitz 
function. 

On R"^ with coordinates xi,X2,X3,X4, we consider the following vector fields: 

(3.5) X2 = d2 + xids + ^94, 

X3 = d3+ Xid4, 

Such vector fields form a Lie algebra which is 4-dimensional. Their only non-trivial brackets 
are 

(3.6) [Xi,X2] = X3, [Xi,X3] = X4. 

Therefore, such an algebra is isomorphic to the Engel Lie algebra. Using the general theory 
of (nilpotent) Lie groups one can prove that there exists a (unique) product on for which 
the vector fields Xi , X2 , X3 , X4 are left-invariant (and therefore a basis of the Lie algebra) . 

Actually, the coordinates for the Engel group that we are using are called the exponential 
coordinates of the second kind. Namely, if Xi, X2, X3, X4 are a basis of the Lie algebra that 
satisfies (13. 6p . then the map 

{xi,X2,X3,X4) exp(x4X4) exp(x3X3) exp(3;2X2) exp(xiXi) 

is a diffeomorphism between and the Engel group. Moreover, the vectors Xi, ^2,^3, X4 
are pulled back to R^ to the vector fields as defined in (13. 5p . 

Recall that in a Lie group G there is a differential geometric interpretation for the product 
between an element p (z G with the image exp{tX) of a multiple of a left-invariant vector 
field X via the exponential map. Indeed, one has the formula 

(3.7) p-exp(tX) = (p), 
where ^^-^(p) denotes the fiow of X after time t starting from p. 
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The aim of the section is to study those sets that are invariant in the direction of Xi and 
are monotone in the direction of X2. Namely, let C be an open set (we always take 
the Lebesgue representative), we say that E is X2- calibrated if the following two properties 
holds: 

Xi-invariance: if p E then, for any t € K, pexp{tXi) S E; 

X2-monotonicity: for all p € R^, the set € K : pexp(tX2) S E} is an open half-line 
of the form (T, +00) for some T € { — 00} U R U {+00}. 

Therefore, if E is an X2-calibrated set then E has constant normal equal to X2, i.e., 
X2tE ^ and XiIe = 0. Viceversa, by Lemma 13.21 and Observation 13. 3|. any set E with 
normal X2 admits an X2-calibrated set E that is equivalent to E. 

By formula (I3.7p . we can calculate a product p ■ exp{tX) without knowing an explicit 
formula for the product. Let us consider the two cases when X is Xi or X2 as above. 
Regarding the flow of Xi, we need to solve the ODE 

f 7(0) = P 

(3.8) 

[ j{t) = (Xi)^(,). 

Writing 7 = (71,72,73,74) and using the definition of Xi, the second inequality becomes 
(7i(*),72(i)>73(*),74(i)) = di = (1,0,0,0). Integrating, we have 

71 (i) =Pi+t, l2{t) = P2, isit) = P3, 74(i) = P4- 

Thus, 

p-exp(iXi) = (pi +t,p2,P3,Pi)- 
Regarding the flow of X2, we consider the ODE 
7(0) = p 



7(t) = (X2)^(t) = (0,l,7i(t),(7i(i))V2). 

7l(0=Pl> l2{t) = P2 + t, -fsit) = P3 + Pit, J4{t) = Pi+plt/2. 



(3.9) 

Integrating, we have 
Thus, 

(3.10) p-exp{tX2) = ipi,P2 + t,P3+Pit,P4+pit/2). 

Thus we replace the previous definition: 
Definition 3.11 (X2-calibration). An open set C R^ is called X2- calibrated if 

i) : \ip(^E then, for any t G R, p + (t, 0, 0, 0) G E; 

ii) : for all p € R^ there exists T G {—00} U R U {+00} such that 

{t € R :p + (0,i,pii,pfV2) ^ E] = (T,+oo). 

Since the set E is assumed to be open, condition ii) is equivalent to the following condition: 
ii') : 

p G i > ^ := p + (0, t,pit,plt/2) G E 
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Example 3.12. Let (7 : R — )■ R be a non- increasing and upper semi-continuous function. 
Consider the set 

E := {x e : X2 > gix^)}. 

Since g is assumed upper semi-continuous, then E is an open set. Then we claim that the 
set E is X2-caUbrated. Indeed, such a fact can be seen as a consequence of Proposition 
13.211 from next section in which we give a characterization of sets with constant normal. 
However, we present here a direct and detailed proof of such a claim. Property i) is obvious, 
for such an E, since in the definition of E the variable xi does not appear. 

Let us show property ii'). lip € E, thenp2 > diPi)- Now, if t > 0, we have that p2+t > P2 
and g{pi +Pit) < 5(^4), being g non-increasing. Thus, P2 + t — g{pA +Pit) > P2 — gipi) > 
and sop + (0, t, pit, pjt) G E. QED 

Now we provide the result that a set of finite perimeter with constant horizontal normal 
is, in suitable coordinates, the upper graph of an entire Lipschitz function. 

Lemma 3.13. Let E C R'^ be any an open Xi-invariant and X2-monotone set. Denote 
by R the extended real line, i.e., R := {—00} U R U {-|-oo}. Then there exists a function 
G : R2 ^ R such that 

E = {x : X2> G{X3,X4)}. 

Proof. For each X3,X4 € R, define G{x3,X4) := inf{3;2 : {0, X2, X3, x^) € E}. Here inf{0} = 
-|-oo. Whenever such an infimum is finite, then it is not realized, since E is open. Since E 
is X2-monotone and (0, X2, X3, X4) • exp{tX2) = (0, X2 + t, X3, X4), we have that 

E n ({0} X R X {xs} X {xi}) = {0} X (G(x3, X4), +00) x {X3} x {x^}. 

For any x € R^, since E is Xi -invariant, we have that 

XGE (0, X2, X3, X4) G <;=^ X2>G(X3,X4). 

The upper semi-continuity of G follows because E is open. □ 

Lemma 3.14. Let G be the Engel group in exponential coordinates of second kind with Lie 
algebra as in (j3.5p . Let E d G be an open X2-calibrated set. Assume that there exists p € E 
such that p + (0, R, 0, 0) G E. Then 

{x G R^ : X4 > P4} C E. 

Proof. Let x G R^ with X3 7^ ^3 and X4 > p^. Set s := X3 — P3, which is nonzero, and 

t := , which is positive. By the particular assumption on p, we have 

X4 -P4 

(pi,X2 - t,p3,p4^) G E. 

By Xi-invariance, 

{s/t,X2 - t,P3,P4) G E. 

By X2-monotonicity, 

(S S 5^ \ 

-,X2-t + t,P3 + -t,P4 + -^t \ eE. 
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Explicitly, 

-,X2,P3 + S,P4 + y I = I -,X2,P3 +X3 - P3,P4 + S ^2 ) = [-^^X2,X3,X4 j e E. 

By Xi-invariance, 

{xi,X2,X3,X4) £ E. 

□ 

Observation 3.15. The previous lemma is saying that the fuction G describing E has the 
property that the closure of the level set at — cx) is a half-space orthogonal to X4. With 
an analogous argument we can actually prove the stronger statement: if (^1,^3,^4) — > 
{pi,P3,P4:) as n — )• 00 and G{pi,p'^,p2) — >■ —00 then on the half-space {x^ > p^} the 
function G must take the value —00. We skip the proof of this statement, since it will easily 
follow from the properties of the set G described in Example 13.341 

Definition 3.16 (Partially Lipschitz map). Let G : R'' ^ R, v £ , and L > 0. We say 

that G is partially L-Lipschitz along v if, for all t > and x £ R^, one has 

G{x + tv) < Lt + G{x). 

Notice that in the above definition we only have a condition for positive t and also for 
the difference G{x + tv) — G{x), not for the absolute value. Example of partially Lipschitz 
maps are the monotone maps. Indeed, every nonincreasing function G : R ^ R is partially 
1-Lipschitz along v, for all L > and all > 0. 

Lemma 3.17. Let G : R"^ ^ R be such that the set E = {x e R^ : X2 > G{x3,X4)} is 
X2-monotone. Then G is partially 1-Lipschitz along any vector [a,a'^/2), with a G R. 

Proof. Fix X3,X4 G R. Assume G{x-3,X4) 7^ +00, otherwise there is nothing to prove. Take 
X2 > G(x3,X4). Thus (a, a;2, X3, X4) G E. Since E is X2-monotone, we have that, for all 
t > 0, 

(a, X2 + t,X3 + at, X4 + —t) G h. 
So X2 + 1 > G{x3 + at,X4 + a^t/2), for all t > 0. Letting X2 G{x3,X4), we get 

G{x3, X4) -\-t> G{x3 + at, X4 -|- a^t/2), 
which ends the proof. □ 
As a consequence we can get the following corollary, i.e. Theorem 11.51 

Corollary 3.18. There exist coordinates in which the set E of constant normal X2 can be 
expressed as upper-graph of a globally Lipschitz function ofR^. 

Proof. From the previous lemma, for any direction v = {vi,V2) in Rxg,x4 with |w| = 1 and 
V2 > 0, the function G is ^^^^ partially Lipschitz along v. So at every point y on the graph 

of G there is a cone-shaped domain y -\- {(xi, X2, X3, X4) : X4 > 0, X2 > ^2x! V ^3 + ^4} 
is contained in the upper graph of G. 

Remark that the cone is independent of the point y on the graph of G, it is just moved via 
(euclidean) translation. By suitably rotating coordinates, we can make therefore E to be the 
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upper-graph of a globally Lipschitz function: namely we have to choose a graphing direction 
that lies in the interior of the set {{xi, X2, x^, X4) : x^ > 0, X2 > ^2x1 V^3~^ ^li- '-' 

3.3. The complete class of examples of sets with normal X2. We present in this 
subsection a characterization (as well as some examples) of sets with normal X2 in the 
model of the Engel group that we have used above. 

We recall a few facts on BV functions, with reference to jGMS98l pages 354-379]. 

Let u be an Lj^^ function on R"; the subgraph SU of n, i.e., the set {{x,y) € R" x R : 
y < u{x)}, is a set of locally finite (Euclidean) perimeter if and only if u is BVioc (Thm. 1 
page 371). 

Let n : R" — >■ IR be Lj^^. Define the approximate limsup and liminf at x G R" respectively 
as follows, where for any t € R we use the notation Ut^u '■= {x G R" : u{x) > t} and 
Lt,u := {x G R" : u{x) < t}: 

u^{x) := sup{i G R : the n-dim. density of the set Lt^u at x is 0}, 

U-{x) := inf{t G R : the Ji-dim. density of the set Ut^u at x is 0}. 

When u^{x) = u-{x) we say that x is a point of "approximate continuity" for u. The 
set J of points where the strict inequality u+{x) > u-[x) holds is the "jump set" of u. 
Then we have (see |GMS981 pages 355]): the set J is 'H"'~^-measurable and countably 
"H^-i-rectifiable. 

The term "jump set" is justified by the result we are about to recall. Denote, for x G R" 
and Vx G 5"""^, the half-space {?/ G R" : (y — x^v) > 0} by E~^{x,v). Analogously denote 
the half space {y G R" : (y — x, v) < 0} by E~{x, i'). 

For 'H^~^-a.e. x in J there exists a (unique) Ux G S^~^ such that it holds: 

av^iuiy^x,y&E+{x,u)u{y) = u+{x) and aplim^^^^. ,,)n(y) = n_(x). 

The notion of approximate limit here (see jGMS98[ pages 210]) is meant as follows: 

for all e > the set {y G £'"^(x, 1^) : \u{y) — u+(x)| > e} has n-dim. density at the point 
X. Analogously for all e > the set {y G E~{x,i') : \u{y) — u^{x)\ > e} has n-dim. density 
at the point x. 

Then we can improve our knowledge of J with the following statement ( |GMS98l pages 
355]): the set J is '^"■"^-measurable and countably '^"■"^-rectifiable; moreover on J we have 
that the approximate tangent space (in the sense of geometric measure theory) exists for 
■^"■"^-a.e. X and is given by the orthogonal to Ux- 

Regarding the distributional derivative Du of the BVioc function u : R" — )■ R, we know that 
it is a locally finite measure (by definition). Setting D^^^u := Du\_J and Du := Du — D^^^u 
we are going to use the splitting Du = Du + D^^^u. The measures D^^^u and Du are 
mutually singular. There exists (see [AFPOOj or [GMS98j ) a further splitting of Du into an 
absolutely continuous (w.r.t. Lebesgue measure) part and a "Cantor part", but we are not 
going to need it for our purposes. The measure D^^^u is just (u+(x) — U-{x)){T-0'\—J) ® Vx- 
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By recalling Theorems 2 and 3 on page 375 of |GMS98) we will now see how to express 
the distributional derivative Dlsu of the characteristic function Isu, for u € BVioc, in 
terms of Du. 

We split 

Dlsu = D^^^ lsu + D^'^°^'^ Isu, 
where D^i^ Isu := Dlsu\-{J x K) and ig^ := Dlsu - ^^^'^ ^su- 

Let denote the coordinates for R" x R. Then it holds, for _d{c°"*) Isu'- 

/j^icont) ^\ (^(^.^y)) ^ f <j){x,u+{x))D,u for any G C;?°(R"xR) and i e {l,2,...n}, 

(3.19) tsu) iH^,y)) = - f^^ <Pix,u+{x))dx for any <^ e C7~(R" x R). 

Regarding the jump part we have that in R" x [R 

(3.20) Z)(^)lsc/ = CH"LF)^(i/,,0), 

where F = {(x, y) G R" x R : x € J, U-{x) < y < u^{x)} and the vector is the normal to 
J in R". 

We are now ready to prove: 

Proposition 3.21. In our model of the Engel group a set has finite perimeter and constant 
horizontal normal X2 if and only if it is of the form {x2 > G{x3,X4,)} for an upper semi- 
continuous function G : R^ — ?• R with the following properties: 

(i) : the closure of {{x^, X4,) : G{x3,X4) = —00} is a half-space of the form {(x3,X4) : 
3^4 > b} for some b G R; 

(ii) : the restriction of G to the open set g := R2 \ {{x3 , X4) : X4 > 5} \ {(x3,X4) : 
G(x3,X4) = +00} is BViociG); 

(iii) : G satisfies the following partial differential inequalit'^ on Q: for all h € C'^{Q) 
such that h > 0, it holds 

(3.22) {{d3G,h)f + 2{dAG,h){C^,h) < 0. 

Here C? denotes the Lebesgue measure on R^ and { , ) denotes the pairing of distri- 
butions and smooth test functions. 

^This is the distributional analogue of the inequality (daG)^ + 2diG < in the case that G is a smooth 
function. Indeed, assuming {d^G)^ + 2d4,G < 0, for any h G C^(Q) such that h >0 and J ft = 1, we have 

J {d3Gfh + 2 J diGh< 0. 

Jensen's inequality applied with respect to the measure of unit mass h dC^ yields 

(^j d-sGhj < J {dzGfh. 

On the other hand, by assuming (|3.22p and using it on a sequence of test functions hn having unit integral 
and converging to the Dirac delta at a point, we pointwise obtain the inquality {d^G)^ + 2d4,G < 0. 
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Observation 3.23. The inequality ()3.22p can be equivalently expressed by requiring that, 
for any h E C'^{Q) such that /i > and / /i = 1, it holds 



We first show the following 

Lemma 3.25. Let G : G d V'^ ^ U he as in Proposition \3.21\ and be J C R'^ its jump part. 
We take G to be a function of the variables xs and X4 and we will denote by 83 (resp. 84) 
the partial derivative, which is a Radon measure, with respect to the variable X3 (resp. X4). 

The partial differential inequality i3. 22\) splits into (and actually is equivalent to) 



function. The second in 1^3. 26\) is equivalent to saying that J has normal that is parallel 
to the Xi-direction for Ti^-a.e. x G J. 

Proof of Lemma \3.25[ We shall prove that (|3.22p yields the two inequalities in (13.26P . 

By definition of ^""^-rectifiable we have J = U^2^^fi{Ki), where the Kj's are compact 
sets in R and /j's are Lipschitz functions from R to Q. We can assume the union U^^fi{Ki) 
to be disjoint. Fix any e > 0: for each N we can find an open neighbourhood A^^^ of the 
compact set ^^ifi{Ki) such that C'^{An^^) < e. This is achieved by taking neighbourhoods 
of each fi{Ki) having measure at most ^ and taking their union from i = 1 to i = N . The 
fact that we can find an open neighbourhood of fi{Ki) having arbitrarily small area is a 
consequence of the fact that fi{Ki) has finite "^^^-measure. 

Choose now, for and e fixed, a smooth bump function 4'N,t that is identically 1 on the 
compact set [J^ifi{Ki), identically outside of ^at^^ and takes values between and 1. 

Choose any h G C^{Q). The partial differential inequality (|3.22p used on the function 

/iAr,e := hipjy^f: reads 



(3.27) i{d3G,hN,e) + {8i'^G,hN,e)f + 2{d^G,hN,e){C^hN,e)+2{8i'^G,hN,e){C\hN,e) < 0. 



(3.24) 





Keeping N fixed and letting e — > we get that 



where the convergence holds since Vjf^-^fi{Ki) = ne>o^Ar,e) ^sG is a Radon measure and 
the sets Atv,^ are bounded. This, together with an analogous convergence for d^G and >C^, 
gives that for fixed and e — ?> 0: 
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Let US now look at the remaining terms in ()3.27l) . namely those involving the "jump 
parts". Denote by v^^j^ the measure 4'^GL(u^i/,(K,)). In the same fashion let v^m := 
di^^u\_{ufLji{Ki)). Remark that i^3,N ^ d^^^G and i/4,7V ^ Sj^G as iV ^ oo. 

For a fixed we get moreover (recall that ipN,e = 1 on u'iLifi{Ki)) that, as e — )• 0: 

h), {di'^G,hN,e)^ {va^n, h). 
So we can send (j3.27p to the limit for e — ?> and get 

(3.28) (Ktv,/!))' <0, 

which holds for every h > 0. Using the convergence of measures u^^n d^J^G as N 

we obtain that S^^^G = 0, as in ()3.26p . This equivalently means that J has a normal v 
always parallel to the X4 direction. 

In order to get the first inequality in (|3.26p we can use an analogous argument, this time 
using 1 — ipN,e instead of ipN,e- D 

Observation 3.29. the condition on the shape of J is actually equivalent to (J \ U^^-Bj) = 
0, where each Bi is a Borel subset of a line parallel to X3. 

Observation 3.30. It is easily seen that, for G G BVioc, equations (|3.26p are actually 
equivalent to ()3.22p . 

Proof of Proposition \3.21[ As we saw in Lemma I3.13| every set having locally finite hori- 
zontal perimeter and constant horizontal normal equal to X2 is the uppergraph of a function 
G : — 7> R of the variables (xs^x^). Such G is upper semi-continuous and by Lemma 13.141 
the closure of the level set at —00 is a (closed) half-space in the direction X4. Such a G will 
then be Lj^^ on the open set G (see observation 13 . 15 p . 

We have moreover seen that E has Lipschitz boundary (in the Euclidean sense) when we 
choose suitable coordinates (Lemma I3.17p . This makes it a set of locally finite Euclidean 
perimeter; thus, since being of locally finite Euclidean perimeter is a notion which is in- 
dependent of coordinates, going back to the original coordinates the function G must be 
BVioc on g. 

We thus need to prove that, for G as in assumptions (i) and (ii), the set E := {x G R'^ : 
X2 > G{x3,Xi)} is X2-monotone if and only if G satisfies (|3.22p . 

Regarding X2-monotonicity, we split the derivatives dx^ 1_e in the "approximately contin- 
uous part" and the "jump part". 

We can compute, on the "approximately continuous part" (R^^ \ {R^i x J)) x Rx2) 
the horizontal normal to 1e as follows: for any non-neg ative h G G~(R^) it holds (from 



2 \ cont 



(^2 + X^ds + ^di^ 



(h) 
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(3.31) 



h{xi,G{x3,X4),X3,X4:) 



1 - Xi{d3G){x3,X4^) - Y(a4G)(x3,X4) 



We now consider J xR and (recall ()3.20p ) here we have d2lE = 0. Let further (a, 6) be 
the vector (Sal^, (941^). Then 



/ x'^ \ ^-"^ 

(3.32) ld2 + x^d3 + ^dA 1e 



xi{u+ - u')a{rL^^J) + ^{u+ - u-)b{V}\-J). 



Altogether, summing the two expressions in (j3.3p and (j3.32p . we get the expression for the 

^2 



X2-derivative of l^;. The condition of X2-monotonicity, i.e. that 



d2 + xid3 + -j-di ) 1e 



be positive for any /i > and for any xi is fulfilled if and only io for any /i > 0, the po. 
nomial in xi 



{h) 



1- 



j hdC^ - Xl{{d3G){x3,Xi),h) - ^{{diG){x3,Xi),h) 

is always positive and 

xia + ^b \ > 0. 

The first is in turn equivalent, since such a polynomial has value 1 for xi = 0, to the 
discriminant {d3G{h))^ + 2diG{h){C'^ ^h) being nonpositive. 

The second is satisfied if and only if a = 0, 6 > 0. The vector (a, b) is, on the other 
hand, the normal v to the jump set J C of G: so the X2-monotonicity is equivalent to 
J being a countably 7^^-rectifiable set with constant normal in the direction 3:4, as in the 
assumptions. □ 



We give now some explicit examples of sets having locally finite horizontal perimeter and 
constant horizontal normal equal to X2 in our model of the Engel group. The first one is a 
generalization of Example 13.121 

Example 3.33. Let (7 : R — )■ R be a non-increasing and upper semi-continuous function. 
Take K € (0, 00) and a non-decreasing function / : [R — ?> R Lipschitz continuous with 
Lipschitz constant < The set 

E ■={x ^R"^ ■.X2> f{Kx3 - X4) + g{xi)} 



"^Indeed the two measures in (|3.3|l and p.32p are mutually singular and the inequality 
xl \ 1 

82 + xidz + —di] 1e (h) splits in the two corresponding inequalities for the two measures 

d2+xid3 + -^d4j 1e and yd2 + xid-i + -^d4,j 1e- This is proved using bump functions as done in 
the proof of Lemma 13.251 
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is X2-calibrated. We can easily see this fact as a consequence of Proposition 13.211 by com- 
puting 

((93G)(X3, X4))' = K\f'iKxs - X4)f, (54G)(X3, X4) = -f{Kx3 - X4) + d^gixi) 
so that 

{{d3G){x3,X4)f + 2(94G)(X3,X4) = {K^'iKxs - X4) - 2)f'{Kxs - Xi) + 2dig{xi) 

< 2^45 (X4) < 

by the condition on the Lipschitz constant of / and by the monotonicity of g. 
Example 3.34. The set 



C := Ix eR^ : X2> 0,X4 > 0,X2 > -^1 
I 2j;4 J 



is X2-cahbrated. In this case we have G = +00 for 2:4 < and G = ^ for X4 > 0. 
Again, making use of Proposition 13.211 we can compute, for X4 > 0: 

2 2 

{{d3G){x3,X4)f + 2(94G)(X3,X4) = ^ - ^ = 0. 

X^ x^ 

We remark here that we get because C is a sort of "extreme case", in the sense that, 
taken any X2-caUbrated set if p € dE then p + C must he in the interior of E. This fact 
win be discussed in detail and play an important role in a subsequent work. 

Let us prove the previous claim. Assume that E is Xi-invariant and X2-monotone and 
let p = {pi,P2,P3,P4:) G E. Then the whole line i = {{pi +0,^25^3,^4) : a S K}, belongs 
to E. Now the X2 -monotonicity means that we can flow from any point in i for positive 
times t along X2 and we remain in E. Writing this down explicitly we get 

(pi + a,p2,P3,P4) + (o,t, (pi + a)t, ^Pl^^t) € E for any t > 0,a G R. 



By using Xi-invariance again we get that 

{Pi,P2,P3,P4)+ (0, t, {pi + a)t, ^"'^ t] eE foranyt>0,aGR. 



The points (t, {pi + a)t, with t > and a S R describe the surface {2xz = 

y2 : X > 0,z > 0} in R^. This means that whenever E contains p then it must contain 

^2 

the surface p + {(xi, X2, ^3, 0:4) : X2 = 2x1^^"^ ^ 0,2:4 > 0}. But recalling that E is 

an upper-graph in the direction of the X2-coordinate we get that E contains the whole 

2,2 

p + {(xi, X2, X3, X4) : X2 > ■2ii,X2 > 0, X4 > 0}, which is exactly p + C. 

4. Sets with normal X2 as intrinsic graphs 

In this section we look at the expression of E (a set with constant normal X2) as subgraph 
of a function when we use as "graphing direction" the flow lines of an horizontal vector field. 
The most natural choice would be to use the flow of X2 as "graphing direction" , as we are 
about to explain in the next subsection. 
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4.1. Intrinsic graphs in the direction of the normal. Let W C he the set of points 
with second component equal to zero, 

W:= {p € : P2 = 0}. 

One can show that is a subgroup of with respect to the Engel structure. Indeed, to 
see this, it is enough to observe the following two facts. First, the vector space spanned by 
the vector fields Xi, X^, form a Lie sub-algebra. The second fact to notice is that the 
span of such vectors is tangent to W. Thus is a subgroup whose Lie algebra has basis 
Xi, X-^, X4. From the algebraic viewpoint, the subgroup 1^ is a complementary subgroup of 
the one-parameter subgroup tangent to the vector field X2. From the geometric viewpoint, 
for each p € K^, the 3-dimensional plane W intersects the line t 1— >• pex.p(tX2) in one and 
only one point. Indeed, by ()3.10|) the second coordinate of pexp{tX2) is p2 + t, which is 
zero when (and only when) t = —p2- We conclude that the space R'^ can be parametrized 
by the following map 

^ -.W xR^R^ 
{p,t) ^ pexp{tX2). 

Assume now that £^ C is a X2-calibrated set. We plan to write E as an upper graph of 
a function. By the definition of X2-calibration there exists a map p 1— )■ T{p) from R^ to R 
such that 

{t : pexp{tX2)) eE} = {T{p), +00). 
Restricting such a map T to W. We get that 

E = {^{p, t) :peW,t> T{p)} 

= {^{p,T{p) + t) ■.peW,t>o}. 

= {{put,P3 + Pit, PA + plt/2) :p£W,t> T{p)}. 

Let us study the map p 1— t- T{p) from to R, in the examples Example 13.121 where 
E := {x ^ R^ : X2 + 5(^4) > 0}, with g non-decreasing and upper semicontinuous. The 
value T{p) is the lower value T such that 

T + gipA+plT)>0, 

since on W we have p2 = 0. Restrict the map T to W H {pi = 0}, so 

T + 5(p4) = 0. 

In conclusion, T : 11^ ^ R is as much non-regular as g is. In particular, there are examples 
of non-continuous function T. We can summarize the last discussion in the following fact. 

Theorem 4.1. There are examples of X2- calibrated sets E with the property that, when 
they are written as upper graphs in the direction of X2, the function for which they are 
upper graphs is not continuous. 
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4.2. Graphs in other horizontal directions. For a set E with constant normal X2, the 
previous example has shown that there can be a lack of continuity for the intrinsic graph 
representing the boundary of E when we choose the flow lines of X2 as graphing directions. 

We might however still look at what happens when the boundary of E is represented as 
an intrinsic graph using different horizontal flow lines as graphing direction: namely let us 
observe the flow lines of aXi + X2 for a > 0. 

First of all we need to write down, analogously to what was done in (j3.9p . the flow of 
aXi + X2 in the model of the Engel group considered so far. What we get is that the flow 
line starting at {pi,P2,P3,P4:) is 



(4.2) (^p^+at,p2+t,p3+pit + ^t^,pi + ^t + ^t^ + ^t^^ . 

We are going to show now that this intrinsic graph might fail to be Lipschitz. To see this, 
we will consider the set in Example I3.34[ 

Theorem 4.3. When the set C := |x G : 0:2 > 0,X4 > 0,a:2 > ^| represented as 
intrinsic upper graph in any horizontal direction aXi + X2, the function for which it is 
upper graph is not Lipschitz. 

Proof of Theorem \4.3\ The intrinsic function T{p) yielding the upper graph is a function 
T : W ^ R, where W := {p ^ : p2 = 0} as before. The value T{p) is the infimum of the 
times t for which the flow line starting at p is inside E. 

Let us restrict the attention to points in W with pi = ps = 0. The flow lines are then 

at, t, ^t^,pi + ) and we must see when the flow line enters the set C. The value T{p) 



for p4 > is clearly 0, while for p4 < it is the solution of the following equation in t 

2t (p4 + y t^) = 
with the constraint that p^ + ^t^ > 0. 



Solving this equation we get ^t^ = 2|p4|, i.e. t = -y/^^^, which fulfils the constraint 



r,2 



P4 + ^t > 0. So we have that, restricting to pi = Ps = in W , the function for which 

C is upper-graph is y'^^^^ for p4 < and identically for ^4 > 0, so it is not Lipschitz 
continuous. □ 

It is therefore necessary to use non-horizontal directions as "graphing direction" (as done 
in corollary I3.18P in order to see the Lipschitz continuity of the function describing the 
boundary of a set with constant horizontal normal. 

The previous proof leaves however still open the possibility for the intrinsic graph in 
the direction aXi + X2 to be Lipschitz with respect to the intrinsic Carnot-Caratheodory 
distance on W (in particular Holder continuous for the Euclidean distance), as we are about 
to discuss. 



SETS WITH CONSTANT NORMAL IN THE ENGEL GROUP 



23 



Lemma 4.4. When a X2-calibrated set E is represented as intrinsic upper graph in any 
horizontal direction aXi + X2, with a ^ Q, the function for which it is upper graph is 
Holder- continuous. 



Proof of Lemma 4-4' We know that, whenever p S dE, then the set p + C is contained in 
the interior of E, where C is the set in Example 13.341 In order to prove the theorem it 
therefore suffices to show that C itself is, in the direction aXi + X2, the upper-graph of a 
function that is Holder-continuous at the origin. Analogously to what was done in proof of 
Theorem 14.31 the intrinsic graph is the lowest value of t such that 



12 



- psat^ + 2pit -pi>0, 



for P4 < with the constraint that ^4 + ^t^ > 0. 

At i = the polynomial is clearly negative. Let us substitute the value t = K max{ - 
for some constant K > Q. 



Then we get, in the case ylpsl > 
2 



12'^' 



K P3a|p3|-2K|p4||p3 



m\ 



i^2a|p3|2 + 2K|p3|2|p3 



+ |P3P 



and this expression is positive if K is chosen suitably large. 
In the remaining case y^lpsj < \/[p4l we obtain 



K'^pj,a\pi\i -2K\pi\\pi 



K'^a\p, 



4 4 4 

|3 + 2K\p4\i + \pi\3 



and again this is positive if K is chosen suitably large. 

Altogether we have shown that for any p & W there exists a value T(p) yielding the 
function describing C as intrinsic upper graph in the direction aXi + X2 and T{p) is below 
if max{ y^IpsIi \/|p4|}- In other words T{p) at the origin has at least the Holder regularity 

□ 



of the function K max{ |p3 1 , vIjmI}- 



As a corollary of the previous proof we can state 

Theorem 4.5. When a X2-calibrated set E is represented as intrinsic upper graph in any 
horizontal direction aXi + X2, with a ^ 0, the function for which it is upper graph is 
intrinsically Lip schitz- continuous. 



Proof of Theorem |^.5[ In order to prove the theorem we must prove that whenever p E dE 
then there exists an intrinsic cone at p whose interior part is all contained in E. In the 
proof of the previous lemma we have seen that the uppergraph of K m.ax.{\/\p^\, \/\p^\} is 
all contained in C, and we know that p -|- C is all contained in E. 

Now the upper graph of K max{ ^/jpsj, \/[p4|} contains an intrinsic cone with respect to 
the Carnot-Caratheodory distance on VF, thanks to the ball-box Theorem (see |LD10] ). □ 
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5. Regularity in filiform groups 

A stratified group G is said to be a filiform group if the strata Vj of the stratification 

Lie(G) = Vi®...®Vs 

of the Lie algebra Lie(G) are such that dim 14 = 2 and dim Vj- = 1, for j = 2, . . . , s. Here s 
is the step of the group. 

One can easily show that there exists a basis Xq, . . . Xg of Lie(G) by vectors with the 
following property: Xq,Xi G Vi, Xj G Vj, for j = 2, . . . , s, and 

Xi, for j = 2, . . . , s. 



(5.1) [Xo,Xj.i]-..„ 

In general there might be other non-null brackets of elements of this basis. In |Ver70] . 
Vergne gave a classification of all stratified groups. In fact, in |Ver70l Corollary 1, page 93], 
Vergne showed that, in the case the step s is even (so the dimension of the group is odd), 
then there is only one stratified group of step s and it admits a basis for which the brackets 
are all null, except those in ()5.ip . In case s is odd (and the dimension is even), then there 
are only two different filiform groups: one where, a part from (15. ip . all other brackets are 
null and a second one where the only other non-null bracket relation is 

[Xi,Xs^l] = {-iyXs, for / = 1,..., s-1. 

We refer to this two groups as the filiform group of the first kind and the filiform group of 
the second kind, respectively. 

We shall show how the argument for proving the regularity of constant-normal sets in the 
Engel group can be extended to any filiform group of the first kind. 

Theorem 5.2. Let G be any filiform group of the first kind. Let E C G be a set with 
horizontal constant normal. Then E is a Lipschitz domain. 



Proof. Let Xq,...Xs be a basis of Lie(G) satisfying (15. Ij) . As for the Engel group, we 
can assume that either XqIe = or XiIe = 0. Consider first the case XqIe = 0. By 
Proposition 12. 2^ the vector Adexp(jXo)^i is a monotone direction. Explicitly, by (j5.ip . we 
have 



Adexp(tXo)-'^l 



gad(tXo)j^^ = Xi 

Pick s distinct numbers ti, . . . ,tg. Consider the vectors 



tX2 + -X3 + . 



+ 



1 



-X,. 



Adexp(t^Xo)-^l) 



for j 



h 



,s. 



We claim that the vectors Yj are linearly independent. Indeed, it is enough to show that 
the matrix 



1 tg 







2 


s-1 






2 


■ s-lJ 
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has full rank. Equivalently we need 

/I h tl 

det : : : 

\1 tl 



We observe that we are considering a Vandermonde Matrix. Hence such a determinant is 
^i<i<j<s{ti — ij), which is nonzero, since the tj's have been chosen to be distinct. Since we 
found a basis of monotone directions, as for the Engel group, we conclude that the set E is 
(equivalent) to a Lipschitz domain. 

Let us consider now the case XiIe = 0. Applying Proposition [2]2l we get that the vector 
field 

Adcxp(tXi)-'^0 = -'^o - 

is a monotone direction, for all t € R. Thus X2tE = 0. Iterating the use of Proposition 
12.21 and using (|5.ip . we get that all vectors X2, ■ ■ ■ ,Xs are invariant directions. Hence E is 
half-space. □ 
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